Distributed, predictive, dichotomous decision engine for an electronic personal assistant

ABSTRACT

A system, method and user interface are described for providing a personal assistant functionality using a predictive, adaptive, dichotomous (two choices) decision engine that proactively prompts the user for decisions on matters deemed relevant by the decision engine based on past user decisions and activities.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. 119(e) of U.S.Provisional Application No. 61/435,985 filed on Jan. 25, 2011.

FIELD OF THE INVENTION

The present invention is in the field of artificial intelligence. Morespecifically, the present invention is in the field of electronicpersonal assistants using adaptive intelligence to evaluate likelyactions and decisions relevant to the user for the purpose of assistingthe user to organize his or her day-to-day activities.

BACKGROUND OF THE INVENTION

People generally have a need for organizational tools to manage theirbusy lives. Such tools have, in the past, consisted of clumsypaper-based systems, and more recently have begun to make use ofcomputer technology. Today, portable personal assistant devices, such assmartphones and personal digital assistants (PDAs), exist to help peoplekeep track of information, scheduling, appointments, reminders, contactinformation and myriad other bits of data. Such devices can presentprescheduled alerts and reminders to the user. Many of the latestdevices can synchronize their data with remote servers, and receivemessages and instructions from remote sources. However, these devicesgenerally do not have any sort of built-in intelligence. They are unableto adapt to the behavior, environment or needs of the user. The user hasto explicitly instruct the device to perform any specific action. Thislimits the usefulness of such devices as personal assistants, as theuser must constantly monitor the function of the device to ensure it hasall the correct data and has been programmed to perform the correctactions at the correct times.

It should be noted that Patent 61/445,433 includes concepts related tothe invention described herein.

Hands-free computing devices, such as Bluetooth headsets, exist thatallow the user to interact with a computer or mobile device, givingcommands and receiving information. Rudimentary neural interfaces evenexist that can detect commands and responses indicated by the user'sthoughts and emotions. However, none of these hands-free devices iscapable of connecting directly and immediately to a more distributedsource of processing power, such as a “cloud” of networked devicesavailable for computing, and none is capable of adapting its behavior tothe user's current circumstances—environment, status, activity, etc.—orpredicting the user's desires and proactively consulting the userregarding potentially desirable actions and information.

In using a neural interface, the user thinks a particular thought orexpresses a particular emotion to register a particular input, such as“yes” or “no.” An example of such a device is the NeuroSky MindSet.These neural devices, however, can be difficult to learn to use, theyrequire a great deal of concentration to ensure the correct thought isbrought to mind or the correct emotion expressed, and they are quiteslow in terms of how quickly the user can register an input. The levelof concentration necessary precludes use of a neural interface whileperforming other tasks requiring concentration, such as driving a car,holding a conversation or operating machinery. Further, the difficultyof registering an unambiguous stimulus to the interface makes thesedevices useful only for the simplest and least critical applications.These are severe limitations that prevent many potential uses for neuralcontrol devices.

Transcription services exist whereby a user can call a specifictelephone number, speak a message and have the message sent to any of anumber of destinations, including an email address, a blog and themicroblogging service Twitter. The limitation of these services,however, is that the user must call a number and wait for a connectionbefore speaking his or her message, which consumes a great deal of extratime and requires extra focused attention.

Predictive dialers exist that automatically call telephone numbers, playa recorded message, wait for a response from the recipient of the phonecall, whether via keypad entry on the telephone or via interactive voiceresponse (IVR), and then play another recorded message or transfer therecipient to a particular telephone number based on the response. Butthese dialers are fixed in their functionality, cannot ask questionsautonomously and cannot automatically adapt to the user's preferencesand behavior.

Limited data prioritization systems exist, such as Gmail's PriorityInbox, that adaptively sort information based on relevance orimportance. However, these systems are generally limited to specificdomains and do not operate broadly enough to be useful in a truepersonal assistant system.

Numerous services exist that present recommendations (such as productrecommendations) based on historical and “crowd-sourced” data—datagathered from the behavior of large groups of users—such as Hunch.com,Mint.com and Amazon.com. However, none of these services issophisticated enough to support a true personal assistant system.

Systems have been previously described that estimate a user's current“state” in terms of activity and availability, but none of these systemsuse the resulting conclusions to provide the adaptive interaction anddecision-making capabilities of a true personal assistant. Further, somesystems make possible semi-automated scheduling using some level ofautomatic decision-making capability, but these systems are limited inscope and fall short of the qualities desirable in a true electronicpersonal assistant. Other systems exist that make use of distributeddata sources and the user's current context to provide relevantinformation, but these too fall short of a true personal assistant inthat they simply provide information and do not provide thedecision-making capabilities desired of a personal assistant.

Live, remote personal assistant services exist that provide thefunctions of a personal assistant to many customers for a fee, andoperate via phone, email or other communication channels. However, theseservices are just a different way of providing traditional personalassistant services, and still require that a live individual perform theservices. The automation of services is not present, as is desirable ina true electronic personal assistant system. Further, as the customerusually does not know the hired assistant personally, lack of trust andclose communication can present obstacles to effective service.

When being called on to make decisions, executives usually prefersimple, clear proposals, providing all relevant data and requiringapproval or disapproval (or possibly deferral). This reduces the burdenon the executive and increases efficiency. However, existing digitalassistant devices fail to meet this need, requiring a human operator tofind, evaluate and analyze information. Thus, the executive either hasto do the work himself or hire a live assistant to manage his digitalassistant, effectively defeating the purpose.

The present invention has the object of solving these problems.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an electronicpersonal assistant device that assists the user in recalling informationand provides the user with timely reminders and alerts.

It is another object of this invention to query and/or infer the user'sbehavior patterns, on a one-time or ongoing basis, so as to be able topredict desired or necessary actions and present them to the user forapproval.

It is another object of this invention to record the user's behavior andobserve the user's current location and circumstances, and use theresulting data to adapt its operation to best suit the user's needs,predicting as reliably as possible which data the user will need at anygiven time and presenting it to the user.

It is another object of this invention to ask the user dichotomous ormultiple-choice questions—i.e., requiring yes/no or other dichotomousanswers, or questions with a larger number of choices—to determinewhether predictions made as to the user's needs are correct, and adaptfuture behavior based on the responses to such questions.

It is another object of this invention to reduce decisions to simple,direct queries that can be presented to the user, briefly and with allrelevant data, for decision, such that the user is asked to make asimple choice between two or more options.

It is another object of this invention to allow remote storage andprocessing of the user's data, including execution of the adaptation andprediction algorithms used.

It is another object of this invention to allow the remote storage andprocessing unit to initiate dichotomous queries of the user bycommunication via the user's electronic personal assistant device, andreceive the responses to said queries from the device for storage andprocessing, via a phone call, computer network communication or othermethod.

It is another object of this invention to allow the user to delivercommands to an electronic personal assistant device, via speech oranother means, for execution, either directly on the device or via theremote storage and processing unit.

It is another object of this invention to provide the user with avoice-driven interface allowing control and communication solely viaspeech, but also to provide a physical interface for control andcommunication with the system.

It is another object of this invention to detect, either by direct usercommand or by recognition of unique environmental circumstances, the“state” of the user—i.e., whether the user is busy and shouldn't beinterrupted, needs only urgent alerts and reminders, should be freelycommunicated with, etc.

It is another object of this invention to provide a neural interface,using the brain's emitted waves, neurally induced energy effects,muscular impulses or physical movements to detect dichotomous (e.g.,yes/no) or simple multiple-choice answers to questions.

It is another object of this invention to make possible variousdifferent types of user interfaces, including, but not limited to,neural, emotional and physical interfaces.

It is another object of this invention to provide a low-latency userinterface that responds promptly even when using a remote unit forstorage and processing, and that is “always on” and will react quicklyto voice or other commands without any sort of initialization or wake-upcommand.

It is another object of this invention to provide a low-latency userinterface that responds reliably, and will either deliver the userrequest or promptly inform the user that the task will be delayed untilthe necessary resources (such as network connectivity or a functionalsmartphone) are available.

It is another object of this invention to provide an active taskmanagement and reminder system to assist the user in accomplishingdesired tasks in specified time frames.

It is another object of this invention to provide a personal contactservice wherein a meeting with one or more other parties can bescheduled and initiated without direct action by the user, via anyavailable communication channel.

It is another object of this invention to allow integration with livepersonal assistant services and to provide an abstraction layer for suchservices.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a diagram of the components and interconnection of oneembodiment of the personal assistant system described herein.

FIG. 2 is a diagram showing the “always on” aspect of a communicationsinterface device as used in some embodiments.

FIG. 3 is a flow chart for the implementation of the “always on” aspectof the system according to one embodiment.

FIG. 4 is a diagram showing the connectivity of the system to the user'svarious data stores.

FIG. 5 is a diagram showing examples of possible system configurations.

FIG. 6 is a diagram depicting the use of auxiliary “plugin” modules toexpand the functionality of the system.

DETAILED DESCRIPTION OF THE INVENTION

Illustrative embodiments and exemplary applications will now bedescribed to disclose the advantageous teachings of the presentinvention.

While the present invention is described herein with reference toillustrative embodiments for particular applications, it should beunderstood that the invention is not limited thereto. Those havingordinary skill in the art and access to the teachings provided hereinwill recognize additional modifications, applications and embodimentswithin the scope thereof, and additional fields in which the presentinvention would be of significant utility.

Some portions of the detailed descriptions that follow are presented interms of algorithms and symbolic representations of operations on datawithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared and otherwise manipulated. It has provedconvenient at times, principally for reasons of common usage, to referto these signals as bits, values, elements, symbols, characters, terms,numbers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing,” “computing,” “calculating,” “determining,”“selecting,” “displaying,” “sending,” “receiving,” “updating,”“modifying,” “assigning,” “requesting,” “notifying,” “communicating,”“interfacing,” “encrypting,” “decrypting” or the like refer to theactions and processes of a computer or similar electronic device havingprocessing circuitry and components that manipulate and transform datarepresented as physical (electronic) quantities within the system'sregisters and memories into other data similarly represented as physicalquantities within the computer's memories, registers or other suchinformation storage, transmission or display devices.

As generally shown in FIG. 1, a preferred embodiment consists of apersonal electronic device 101, such as a smartphone or PDA; acommunications interface 102, such as a Bluetooth headset or neuralinterface; and one or more remote coordination servers 103. Thecommunications interface 102 serves as the user's primary means ofinteraction with the system, allowing the user 104 to issue commands,receive data and respond to queries from the system. The personalelectronic device 101 serves as a communication link to the remotecoordination server(s) 103 and other services, such as telephony andInternet services, but can also be used as a secondary means of userinteraction with the system. The remote coordination server(s) 103perform the bulk of the system's processing tasks and serve as thecentral management point for the user's data, although some of thesefunctions can be performed by the personal electronic device as itscapabilities permit. The personal electronic device 101 and thecommunications interface 102 are connected to the remote coordinationserver(s) 103 via a communications network 105, such as a computer ortelephony network.

In one embodiment of the present invention, the communications interfaceis a headset, such as a Bluetooth headset commonly used with mobilephones, and the user interacts with the system using voice commands andresponses. The headset is linked to the personal electronic device, inthis case a smartphone. The smartphone in turn connects to a telephonynetwork, the Internet and other such services, and uses telephony and astandard TCP/IP network connection to communicate to a set of remotecoordination servers with a standardized communication interface.

The user can issue a vocal command to the headset, which detects thecommand and forwards it to a software control program running on thesmartphone, headset or other device. A key feature of this embodiment isthat there is no need for the user to press a button or speak a commandto “wake up” the system and prepare it to receive commands. The headsetis always listening for user commands and detects when the user hasspoken an intelligible command that can be executed. This is illustratedin the embodiment shown in FIG. 2: a headset 201 is always listening 202for audio input 203 and processing any commands it detects. However, insome embodiments it is possible to signal the beginning of a verbalcommand by pressing a button on the headset or other communicationsinterface. In one embodiment, as shown in FIG. 3, the headset idles in alow-power mode 301 waiting for a sudden increase in volume 302, and thenrecords and processes the subsequent sounds, looking for a parseablecommand 303. If it finds such a command 304, it passes it to thesoftware control program running on the smartphone 305. This can be donein several ways. In one implementation, the entire recording of the usercommand is passed to the control program (buffered mode). In anotherimplementation, the moment a command is detected, the audio informationis streamed to the control program in a way similar to “cut through”routing, as is known in the art. In another implementation, the audio isalways being streamed to the control program, either continuously orwhenever an audio threshold is exceeded. If the smartphone is notimmediately available for some reason—e.g., its power is off or it isdisconnected from the headset, the headset stores the received commandin a buffer, waiting for the smartphone to become available again, atwhich point it proceeds to send all buffered commands to the softwarecontrol program running on the smartphone. In another embodiment, theheadset itself is doing the command parsing using a control program.Some examples of commands the user could deliver to the system are toinitiate a phone call; pick up a phone call; hang up a call; send apredefined message via email, text message, etc.; pay a bill using aspecified bank account; purchase a specified item; post a status updateto a social networking Web site; order a meal for pick-up from arestaurant.

The smartphone, upon receiving a command, either executes the commanditself or forwards it to one of the remote coordination servers forexecution. Some commands have no need for remote execution, such as arequest to initiate a phone call, and these are executed by thesmartphone itself. Other commands, such as a request for a Web search,data from a Web site or information based on the user's currentgeographical location, requires a connection to a remote service, andwould thus be routed to one of the remote coordination servers forhandling.

The remote coordination server then processes the request and returns aresponse indicating success or providing the requested data. Uponreceipt of this response, the smartphone displays a suitable message forthe user and additionally forwards the response to the headset, whichinforms the user vocally of the result if appropriate.

In a similar fashion, a remote coordination server can originatecommunication to the user. Based on its internal algorithms, the remotecommunication server may deem it necessary to provide the user withinformation or present the user with a query, in which case it sends amessage to the user via any available communication medium. It may placea phone call to the user and vocally deliver the communication, or itmay transmit the communication via the Internet to the user'ssmartphone, which in turn delivers the message to the user vocally or onits screen.

The user can configure the system to deliver notifications when specificevents occur, and the system will monitor the user's data sources toestablish when such events have occurred and deliver a notification viaany available communication channel (or as limited by the user'sindicated preferences) to the user. For example, the system can notifythe user when a particular person is in a particular location; when anyof the user's friends or family are nearby; when a particular personchanges location; when a particular person becomes available online, ason an instant messaging service; when the status of a house alarmchanges; when the status of a remote computer server changes (e.g., theserver stops responding); when a communication is received, such as avoicemail, text or email message; when the price of a security changessufficiently; when particular events occur in an economic market; when aparticular business condition is present; when some event on anassociate's calendar is changed; when an event is scheduled or changedon a user's own calendar; when an event occurs nearby that is relevantto the user; when traffic conditions change in the user's path; when anairline flight changes its schedule. These notifications, of course,require that the system have access to the relevant data, but thesystem, having access to all the user's data stores and sources, as wellas worldwide network access, has access to an immense amount ofinformation. The available information is analyzed based on the user'srequest for notification and based on behavioral information about theuser, as in the case of the general relevance of an event to the user.

In some embodiments of the present invention, the user can answerqueries via a standard neural interface, as is known in the art. Theuser simply dons the appropriate apparatus and thinks or emotes thedesired response to questions posed by the system, the answers beingdetected by the neural interface and passed to the user's personalelectronic device for handling.

In various embodiments of the system, different user interface devicescan be used, in place of or in addition to the headset specified for thepreferred embodiment. Such interfaces can include, but are not limitedto, neural interfaces, face-tracking systems, eye-tracking systems,gestural devices, bodily sensors, interactive voice response systems andvoice control devices. The user can register input via any of a numberof possible methods, including, but not limited to, voice or othersounds; pressing buttons on a keypad, as on a phone; thinking aparticular thought; expressing a particular emotion; tensing a muscle;typing on a virtual keyboard; blinking; moving the eyes; making aparticular face; smiling and/or frowning; winking; sighing; whispering;making a specific motion; raising a number of fingers; shaking ornodding the head; and exercising a “phantom limb,” wherein the useractivates a little-used muscle or nerve; muscular stimuli; or electronicsignals induced in the nerves, brain and surface of the head or body bythoughts or emotions felt by the user. Output to the user can employ anyof a number of methods as well, including, but not limited to, displayof data on a screen or other visual output device; audio cues, whethervocal or not; vibration; neural stimuli; music; or electronic wavesapplied to the body, head or brain that induce thoughts or emotions inthe user.

FIG. 4 shows a few of the possible system configurations. Note thatthese are merely examples and by no means encompass all possible systemconfigurations. As shown in FIG. 4, the user interface device could be aheadset 403/406, a wristwatch 404/407 or a vehicle 405/408. The userinterface could then connect directly to remote coordination servers viathe Internet 402, or could connect via a mobile phone 401 to the remotecoordination servers.

In a preferred embodiment, the system has access to any available datastores belonging to the user. Namely, the system can draw upon any ofthe user's data stored in the “cloud,” such as Web-based email, contactdatabases, data stored on social networking Web sites, online retailtransactions, news feeds, etc. This is illustrated in the example shownin FIG. 5, wherein the personal assistant server 501 has access to (andthus can retrieve data from and send data to) Facebook 502, Amazon.com503, Gmail 504 and 1-800-FLOWERS 505. The system and user interfaceprovide the user with immediate access to all available data stores bysimple command, and suggest relevant data as described below.

In a preferred embodiment of the present invention, the remotecoordination servers implement a degree of intelligence by adapting tothe user's behavior, location and circumstances, and attempting topredict the user's needs. The remote coordination servers have access tothe user's present location via the facilities of the user's personalelectronic device (e.g., a smartphone), and use this location inconjunction with the time and other current data about the user todetermine what data might be relevant to the user, such as informationabout nearby businesses, proximity of associates or friends, tasks theuser needs to perform near the current location, etc. In addition, theremote coordination server maintains an historical log of the user'sactions in using the system for the purpose of discerning patterns andpredicting potential future user actions or needs. Further, the systemcan use an aggregate historical log of all its users to establishcorrelations between users for the purpose of predicting user needs anddesires: when a correlation is found for a given user, the actions andneeds of correlated users can be used to suggest actions and presentdata to that user. This can be implemented using a statistical“crowdsourcing” approach as known in the art. When the remotecoordination server has predicted that the user might need or want toperform a particular action or view a particular bit of information, itpresents this to the user. If the user needs to make a decision about aparticular action, the system presents a yes/no dichotomous (ormultiple-choice) query to the user and then performs or does not performthe action depending on the user's response. If the system presents theuser with an unsolicited bit of information, it might ask the userwhether the information was useful or not, or the user might originatevolunteered feedback on such, which the system will note for future. Theremote coordination then records the result of any such query in itshistorical log for future analysis. This intelligent functionality is inaddition to the usual functions of an electronic personal assistantdevice—namely scheduling events, alerts and reminders, and requestingspecific bits of information for display to the user. Augmenting thesebasic organizational functions, he system can keep track of any tasksthe user needs to perform and continue to remind the user of such tasksuntil the user reports their completion or cancellation.

In some embodiments, the queries are not limited to dichotomous answers,but might allow multiple-choice answers as well, giving the user anumber of options to choose from. For example, instead of simply askingfor a yes/no answer, the system might allow a third option, such as “askme later,” in which the user puts off a decision on the matter. Aspecific example of this usage would be a reminder of the user'smother's birthday: the system would remind the user of the birthday andthen offer a set of options, such as (a) send flowers, (b) send candy,(c) send a gift card with a note or (d) do nothing (in which case theuser might handle the gift herself).

Note that in embodiments where more complex input devices are available,such as computer keyboards, the user's input to the system is notlimited to dichotomous or multiple-choice answers, as described below,but can include specific commands, allowing more precise control ofsystem behavior and control of aspects of the system that are difficultor impossible to control using a simple multiple-choice system.

A key portion of the intelligence of the system lies in determining whatdecisions the user might make for a given situation. Rather than simplyasking the user what to do, the system analyzes its database for theuser and all the user's data sources to determine the most likely set ofchoices the user might make, thus “boiling down” or reducing thedecision to a simple set of dichotomous or multiple-choice questions.Further, for more complex decisions, the system would present the userwith a hierarchy of queries, presented as a series, each query narrowingdown the final decision toward an exact action. For example, the systemmight remind the user of a friend's birthday, then ask if he wants topurchase a gift for the friend from the friend's Amazon.com wish list.If the user answers yes, the system would retrieve an item from the wishlist and, perhaps after a delay, ask the user if he wants to purchasethat specific item. If the user answers yes, the system might ask if theuser wants to gift-wrap the item, and then whether the user wants toinclude a message. If the user wants to include a message, the systemmight transcribe it from dictation by the user, or might choose amessage typically sent by the user on such occasions and present it tothe user for approval. Finally, the system would purchase the gift andhave it sent to the friend as specified by the user.

A similar technique can be used for advertising. In some embodiments, ifthe user so chooses, the system can recommend products based on theuser's current circumstances, activities and known data about the user.For example, if the user is known to be shopping for a gift for his wifein a particular location, the system might recommend a specific item hiswife has indicated that she wants, and indicate a nearby store where itcan be purchased. Such as system can further include provisions toprotect the user's privacy by performing all calculations within thesystem and sending none of the user's personal data to outside parties,such as marketers or other companies.

In some embodiments, the system initially requests that the user answeran extensive survey as to various personal and behavioral preferences.The questions of the survey are intended to define key constraints forthe system to ensure its decisions match the user's usual decisions andbehavior as much as possible. Such a survey can include, but is notlimited to, questions such as: What do you give as gifts? How manypeople's birthdays do you celebrate? Do you celebrate birthdays? How doyou remember people? How do you like to stay in contact with people? Doyou write short or long letters? Do you stay in touch with (a)everybody, (b) close friends, (c) relatives? What type of work do youdo? What is your work style? What faith do you hold? What package ofcustoms do you follow? These and other questions give the system aninitial base of behavioral information that will be expanded upon byanalyzing the user's various data stores and sources, and by analyzingthe user's behavior on an historical and ongoing basis. The systemfurther analyzes the user's past communications, such as via email, textmessage, etc., to establish the user's communication style, interests,activities, relationships, personal and business networks, and otherfactors, semantically parsing the communications as is known in the art.The system differentiates between different types of communication(e.g., personal, work, etc.) to establish different categories ofbehavior, relationships and preferences. From this analysis and fromongoing analysis of the user's data and activities, the system infersother factors, including, but not limited to, who the user's friends,family and associates are; how the user prefers to communicate (viaemail, phone, text message, letter, in person, etc.); where the userworks; the user's work style; where the user lives; what the userprefers to do for leisure; where the user likes to vacation; what typesof products the user likes to purchase for himself and others; theuser's religion and degree of religiousness; and the user's sexualpreferences.

Additional “plugin modules” of survey information can be integrated tofurther customize the system to the user. For instance, a number ofmodules may be implemented that connect the system to external datasources, such as LinkedIn™ or Facebook™, email systems, search engineswith browsing history, and numerous other systems as known in the art.By providing a conduit between the rich contextual data about the userprovided by these systems, the present system of this invention caninfer additional patterns of activity and behavior and intent from theuser. Interfacing with such systems through a Web-based API is wellknown in the art.

Additional modules within the system can be developed. For instance, animproved gift-shopping module could be implemented using an API providedby the present invention. This API allows applications to be insertedinto and run within the invention, or to run externally to the systemand access data from within the system, depending on the privacy levelbeing enforced. This improved gift-giving module would aggregate all themost frequently communicated-with people in the user's life, determinewho the user has given gifts to before, aggregate as many birthdays andholiday preferences from the cultural notes about the respective users(e.g., Christmas vs. Hannukah), and determine any other appropriateholidays from address book entries or CRM systems (children's birthdays,etc.). The module would then provide a Web-, mobile-app-,voice-response-, or application-based interview to the user anddetermine what kind of gifts the user wants to be asked about, how many,who, relevant price ranges, etc. The module would then subsequently, incoordination with the present invention, ask the user to approve ordisapprove matching offers. This module could be tied in with a specificcohort of cooperating online retailers or a gifting service.Advantageously, the system could be almost completely automated, but atthe point of choosing a gift, the model could be designed to reduce thegift choice to a human-executable task, using an approach ofAmazon.com's “mechanical turk” system, where the context of therecipient, sender, sending history, price range and a set of options areprovided anonymously to a human gift-chooser. Or, the module couldinterface to a gift service that has been granted non-anonymous accessto the user's gifting patterns, where a human looks at the context andthe overall gifting budget of the user, and designates which giftsshould be given for a set recipients. These choices, whether implementedin the anonymous or non-anonymous approach, are then presented to theuser for approval in accordance with the invention.

Although this approach is herein detailed for the specific case ofgift-giving, modules of this type can be developed by third parties orby the implementers of the invention, and can be implemented in acompletely computer-automated way, using artificial intelligence andstatistical matching techniques, as known in the art, to providepredicted solutions to user needs, or can be used to provide candidateanswers and solutions to service-providing humans via a Web or voiceinterface (as in scheduling/coordination applications), who then usetheir intelligence to solve the critical portion of the solutionproposal. In this way, the system and any integrated or third-partymodules can provide personal assistance in a highly scalable way,because a given expert personal assistant can have his or herdecision-making ability scaled across hundreds of users needingassistance over the same time period. Or, conversely, the personalassistance needed for a given user can be farmed out scalably to humanslocated in disparate locations, providing culturally and contextuallyrelevant solution proposals for the user for a small transactional costas opposed to an hourly or salary rate.

FIG. 6 depicts a possible embodiment of this “plugin module”architecture. The core processing module 602 of the electronic personalassistant system 601 makes available an interface that can be used byauxiliary services. The core delivers events (e.g., birthdays,information requests, meeting times) 603 to the auxiliary service 604,which is designed to interface with the core. The auxiliary service thenresponds with a query or some information 605 to present to the user608. The auxiliary service can either be an external component providedby a third party 604 or an internal component built into the electronicpersonal assistant system 606 (for instance, in customized versions).Further, the auxiliary service can make use of human interaction tosatisfy its functionality 607.

An additional aspect of this intelligent functionality is that thesystem can adapt its level of “presence” or intrusiveness to suit theneeds and desires of the user. For instance, the user can command thesystem to stop presenting queries, reminders, etc., or reduce itsmessage delivery to urgent communication only. In addition, the systemcan detect, based on its historical logs of user activity and itssensory inputs (location sensors, noise sensors, device state, GPS orlatitude/longitude location detected from a mobile device, nearbywireless transmitters, etc.), when the user is in a location or set ofcircumstances requiring less intrusiveness, such as in a businessmeeting or a movie theater, and adjust its level of intrusivenessappropriately. This adaptation can take into account the user's usualschedule, as recorded in the historical logs, and adjust its behavioraccordingly. Further still, the system can query the user as to thedesirability of a particular communication when it is unsureprobabilistically of the current needs of the user as to its level ofintrusiveness, and store the user's response in its historical logs foruse in making future predictions about the user's needs.

In some embodiments, the system initially surveys the user as to hisusual schedule and when he is willing to be interrupted for queries.Such a survey can include, but is not limited to, questions such as: Doyou want to be interrupted (a) in the shower, (b) on the drive to work,(c) when you are sleeping, (d) when your phone is in silent mode, (e)during meetings? For whom do you want to be interrupted? For what sortsof situations do you want to be interrupted? When do you not want to beinterrupted? What are your daily and weekly schedules? Is there aspecific time each day when you would like to be presented withdecisions for the day? Further, the survey can allow the user to specifydegrees of “interruptibility”—that is, during certain times or givencertain circumstances, the user can specify his preferences on a scalefrom “should be interrupted” to “absolutely must not be interrupted.”This can include such designations as “should be interrupted only forimportant matters” and “ask discreetly whether I should be interrupted.”

When the system has a query or information to present to the user, itfirst queries the user as to whether he is willing to be interrupted,unless it is already known that the user is willing to be interrupted(such as at defined times or immediately after another query) or isunwilling to be interrupted. To contact the user, the system uses allavailable communication channels specified by the user for contact, suchas directly via the system's primary interface device, via phone, viaemail, via text message, via instant message, etc. To preserve privacy,if the system does not have a direct line to the user, as via thesystem's primary interface device, it only presents the user with anotification that it is attempting to contact him and does not includeany of the content of the communication. When the user then responds orrequests the remainder of the communication, it is provided via thedesired channel. Further, in some embodiments, when the system requestsan interruption, in addition to allowing or disallowing theinterruption, the user can respond to the request with indication thathe is only to be interrupted if the request is quick, and the systemwill adjudicate the complexity of the communication and withhold it ifit includes more than a specified amount of information or number ofqueries. Thus an interruption is divided into two actions: (1)determining whether the user is willing to be interrupted or notifyingthe user that information or a query is available, and (2) presentingthe query or information.

As it may occur that the user is unreachable at certain times or undercertain circumstances, or simply does not want to answer questions posedby the system, the system can “save up” messages and unansweredquestions and present them to the user when he or she is next available,or when he or she indicates readiness to receive the backlogged messagesand questions. In addition, the system can predict the user'savailability and willingness to receive messages and answer questionsbased on the historical logs of user activity, and adjust itspresentation of information and posing of questions appropriately.Further, the user can define times he wishes to devote to answeringqueries from the system, such as a particular time every day, and thesystem can majorly present non-urgent queries and other information tothe user at the specified time(s).

One use of the present invention is as a planning aid. In someembodiments, the system keeps track of the tasks the user needs or wantsto complete, and when, and helps the user to complete them by making anydecisions it can for the user, and by presenting reminders and queriesto the user at appropriate times. For example, the system might remindthe user to take vitamins daily, walk the dog in the morning, get an oilchange every 3,000 miles of driving or check the status of a networkserver on an hourly basis. The system, having access to all the user'sdata stores, integrates closely with all of the user's calendars andplanning tools, and conforms to any specific planning methodologies usedby the user. For instance, the system can integrate with such tools asMicrosoft Outlook, Google Calendar, Franklin Planner software and othersuch planning tools, and conforms to such planning methodologies asGetting Things Done and the FranklinCovey planning methods, amongothers, as desired by the user. Further, the system includes its ownplanning interface to allow the user to schedule events and tasks to berecorded in any or all of the user's planning tools. In someembodiments, an initial survey of the user is done to establish commontasks and management methods, asking questions such as: What do you needto get done every day, and when? What do you need to be nagged about?What project management do you need? What micromanagement do you need?Which tasks are (a) important, (b) neutral, (c) unimportant, (d) only tobe done in free time? Do you use a specific planning methodology? Do youneed help getting to appointments on time? Do you like to be frequentlyreminded or told only once? Thus, the system gains an initial profile ofthe user's activities and schedules, and can combine this with datagathered from an analysis of the user's data stores. With a knowledge ofall the users schedules and tasks, as well as the planning methodologythe user prefers and specific preferences about reminders, the systemcan proceed to give the user the needed assistance in ensuring schedulesare kept and tasks are carried out in a timely fashion. This isprimarily accomplished by reminding the user of scheduled events andtasks, with a degree of insistence specified by the user on a general ortask-by-task basis, but also includes the other functions of the system,such as recommending events and tasks for scheduling based on analysisof the user's data stores and sources, and partially performingindividual tasks to the greatest degree possible, then presenting simpledichotomous or multiple-choice queries to the user for decision andacting on the decisions to the degree it is possible.

In some embodiments, the system provides the capability of schedulingand initiating meetings automatically. This proceeds roughly as follows:The user indicates that he wants a meeting with a particular person at aparticular time or within a particular range of time. The system thenautonomously attempts to contact the other person to schedule a meeting.It uses all available communication channels—phone, email, text message,instant message, etc.—to contact the other person and then attempts tonegotiate a meeting time within the constraints defined by the user.This may involve repeated attempts to contact the other person, and theuser can adjust the frequency and insistence the system uses in makingsuch repeated attempts to avoid creating a nuisance. If the systemcannot arrive at an agreement about meeting time, it establishes theother person's preferences as to schedule and presents them to the user,who then either chooses an acceptable time or gives a more preferablerange of times. The system may also ask the other party for a scheduleof availability on an ongoing basis for scheduling of future meetings,if it is determined that the person is someone the user is likely tomeet with regularly. This process repeats until an agreement is reached(or aborts if no agreement can be arrived at). Both parties are notifiedof the final agreement, and the system updates the user's calendar andother planning tools to reflect it. Then, when the meeting time nears,the system asks the user if the meeting should proceed, and if so,reminds both parties of the meeting. At the time of the meeting, thesystem attempts to contact the other person via the preferred channelfor the meeting (e.g., phone). Once the other person has been contactedand is on the line, the system contacts the user to join the meeting, ifhe has not joined already, and the meeting proceeds. The system uses asimilar procedure for meetings involving three or more parties, but hasto negotiate with more individuals to arrive at a meeting time.

Similarly, in some embodiments the system has the capability ofattempting to contact others on behalf of the user. When the userindicates that he wishes to speak with a particular person, the systemautonomously attempts to contact the person via any availablecommunication channels, such as phone, email, etc. In some embodiments,the system can even keep track of the other person's behavior,availability and location over time, and use this historical data topredict possible locations and means of contact for the person. If theperson does not respond and is not available (e.g., offline), the systemwatches the various available communication channels (e.g., instantmessenger, VoIP, etc.) to determine when the other person might beavailable. If the user is also known to be available at that time, thesystem attempts to contact the other person and request a meeting. Theseattempts to make contact repeat at some frequency and level ofaggressiveness specified by the user until the other party is eithercontacted or refuses the contact. Once contact is established, thesystem then contacts the user and connects the two parties.

In some embodiments, the system makes use of large stores of historicaluser data for large groups of users (“crowd-sourcing”) to assist indecision-making Where a user's privacy preferences permit, the systemrecords behavioral data for that user and combines it with the storedbehavioral data of all such users to form a behavioral database. Thesystem then analyzes this database to assist in decision-making, usingstandard techniques known in the art for the prediction of preferencesand other choices.

In some embodiments, the user interface described can use a livepersonal assistant service to do the backend work, instead of or inaddition to one or more remote coordination servers. In this case, thesame control interface is presented to the user, but requests for actionare routed as appropriate to the live personal assistant service forfulfillment. Reminders, information and requests from the live personalassistant service would be delivered to the user via the standard userinterface, effectively hiding the existence of the live service andavoiding any qualms the user might have about using such a service dueto concerns about security or other factors.

In some embodiments, the system provides a safety monitoring andnotification service that keeps track of the user's location andexpected path, and sends a warning if something unusual occurs (i.e.,the user's path deviates outside some range of tolerance). For example,the user could specify a starting and ending time and location for awalk or other journey, and if the user failed to arrive in a timelyfashion, the system might warn a family member or the appropriateauthorities. The user could further specify the intended path, and asignificant deviation from that path would trigger a warning.

As configurability is an aspect of the system, it may be necessary toprovide a more complex interface to the user, such as via the Web or viaa software application running on a personal computer or a smartphone.Such an interface would be used for the initial configuration of thesystem, as via a series of survey modules, and could then be used tomake changes to configuration. In some embodiments, configuration wouldbe possible directly via the dichotomous or multiple-choice interfaceitself, as a means of answering many complex questions aboutconfiguration.

A key aspect of the present invention is that it works around thelimitations of many neural, emotional, physical and gesture-based inputmethods by reducing user interaction to the minimum necessary to arriveat a decision. As many such input methods tend to be clumsy, slow andrequire a great deal of concentration, all but the simplest of inputsare too difficult to register. Registering more than a few simple inputsis extremely difficult due to the necessity to differentiate amongpossible stimuli and the degree of processing necessary to do so.Current computing devices and algorithms are simply not up to the task.Thus, the present invention reduces user input to one or moredichotomous or multiple-choice queries that can be answered relativelyeasily using current mobile or hands-free input devices, with a minimumof concentration and a maximum of speed.

Further, the present invention provides real electronic personalassistant functionality, rather than a simple data store and interfacedevice, as it is capable of making decisions on its own, observingsituations relevant to the user and determining the mostly likelyresponses from the user, then presenting the decision to the user forapproval. Thus, the present invention increases the user's efficiencyand reduces the burden on the user by performing as much work aspossible before requesting that the user make a decision.

Thus, a system, method and user interface have been described forproviding a personal assistant functionality using a predictive,adaptive, dichotomous decision engine.

Therefore, what is claimed is:
 1. A method of operating a personalassistant comprising: (a) accepting a user input command as an audiosignal from a user on a personal electronic device configured to be ableto receive user input commands in audio form without requiring furtherinitialization or wake-up; (b) determining whether the user inputcommand can be fully processed on the personal electronic device; (c) ifit is determined that the user input command cannot be fully processedon the personal electronic device, establishing a communication linkbetween the personal electronic device and a remote device, and usingthe communication link to send user input command information to theremote device for processing only on the remote device by streaming theuser input command to the remote device while an audio threshold for theaudio signal is exceeded; and (d) if is determined that the user inputcommand can be fully processed on the personal electronic device, fullyprocessing the command only on the personal electronic device, whereinthe remote device is a remote server or a smartphone.
 2. The method ofclaim 1 where the user input command is interpreted using speechrecognition.
 3. The method of claim 1 wherein the remote device is aremote server and the remote server sends back a command response whichis delivered or performed by the personal electronic device.
 4. Themethod of claim 1, further comprising: attempting to establish thecommunication link; and if the attempting step fails, buffering the userinput command in a memory of the personal electronic device, such thatthe user input command can be later delivered.
 5. The method of claim 1wherein a software program operating on the personal electronic deviceis configured to connect to one or more user data stores via acommunication link and the software program is configured to retrievedata therefrom.
 6. The method of claim 1 wherein a software programoperating on the personal electronic device is connected to one or moreonline data sources and the software program retrieves data therefrom.7. The method of claim 1, wherein the personal electronic device is aheadset.
 8. The method of claim 1, wherein the personal electronicdevice is a neural interface device.
 9. The method of claim 1, whereinthe remote device is the smartphone, and the processing of step (d) isperformed by another remote server.
 10. The method of claim 1, whereinthe user input command information is audio information.
 11. The methodof claim 1, wherein the user input command information is parsed commandinformation.
 12. The method of claim 1, wherein the user input commandinformation is user interface device information.
 13. The method ofclaim 1, wherein the user input command information is neural inputinformation.
 14. The method of claim 1, wherein the personal electronicdevice is a wristwatch.
 15. An electronic personal assistant systemcomprising: (a) a first personal electronic device configured to be ableto receive user input commands as an audio signal without requiringfurther initialization or wake-up, in communication with one or moreremote devices wherein the one remote device is a remote server or asmartphone; (b) a software program running on the first personalelectronic device that is configured to: accept an input command fromthe user in audio form, determine whether the input command can be fullyprocessed on the first personal electronic device, if it is determinedthat the user input command cannot be fully processed on the firstpersonal electronic device, transfer input command information to theremote device via a communications link for processing only on theremote device by streaming the user input command to the remote devicewhile an audio threshold for the audio signal is exceeded, and if isdetermined that the user input command can be fully processed on thefirst personal electronic device, fully process the command only on thepersonal electronic device.
 16. The system of claim 15 where thesoftware program buffers user commands for later delivery when a neededconnection is not immediately available.
 17. The system of claim 15where the user's interactions with the system are stored in anhistorical behavioral database.
 18. The system of claim 15 where theremote device delivers data to the user via the first personalelectronic device.
 19. The system of claim 15 where the software programis connected to the user's various data stores via a communications linkand retrieves data therefrom.
 20. The system of claim 15 where the firstpersonal electronic device is a smartphone.
 21. The system of claim 15where the first personal electronic device is a Bluetooth device, andone or more of the remote devices is a smartphone.
 22. The method ofclaim 15 where the remote server sends back a command response which isdelivered or performed by the first personal electronic device.
 23. Asystem of operating a personal assistant, the system comprising: aremote server operating a software program configured to receive inputvoice command information from a personal electronic device, thepersonal electronic device configured to be able to receive an inputvoice command in audio form without requiring further initialization orwake-up and to stream the input voice command to the remote server whilean audio threshold for the input voice command is exceeded, the inputvoice command information partially processed by the personalelectronics device, the software program configured to process the inputvoice command information, the software program configured to performcommands or retrieve information based upon input voice commandinformation, the software program configured to deliver the results ofperforming the commands or retrieving information to the personalelectronic device, wherein the input voice command information isprocessed using both the personal electronic device and the remoteserver.
 24. The system of claim 23 where interactions with the systemare stored in an historical behavioral database.
 25. The system of claim23 where the one or more remote servers access information from anonline data source.
 26. The method of claim 25 where the online datasource is one of the following: (a) a remote online database, (b) asocial network, (c) a microblogging site, (d) a website, (e) anhistorical behavioral database.
 27. A method of operating a personalassistant comprising: establishing and maintaining a communication linkbetween a personal electronics device and a remote device; accepting avoice input command from a user on a personal electronics deviceconfigured to be able to receive voice input commands without requiringfurther initialization or wake-up; streaming voice input commandinformation based on the voice input command from the personalelectronics device to the remote device over the communication linkwhile an audio threshold for the voice input command is exceeded;processing the voice input command information at the remote device togenerate a result; and communicating the result to the personalelectronic device.
 28. The method according to claim 27 furthercomprising processing a portion of the voice input command on thepersonal electronic device.
 29. The method according to claim 27 whereinthe communication link to stream a voice input command from the personalelectronic device to the remote device is continuously maintained orre-established.
 30. The method according to claim 27 wherein thepersonal electronic device is a smartphone.
 31. The method according toclaim 27 further comprising operating an API that allows voices servicesto be utilized on the personal electronic device.
 32. The methodaccording to claim 27 wherein the remote device is a remote server. 33.The method of claim 27, wherein the personal electronic device is awristwatch.